86 research outputs found
Orthogonal Matching Pursuit: A Brownian Motion Analysis
A well-known analysis of Tropp and Gilbert shows that orthogonal matching
pursuit (OMP) can recover a k-sparse n-dimensional real vector from 4 k log(n)
noise-free linear measurements obtained through a random Gaussian measurement
matrix with a probability that approaches one as n approaches infinity. This
work strengthens this result by showing that a lower number of measurements, 2
k log(n - k), is in fact sufficient for asymptotic recovery. More generally,
when the sparsity level satisfies kmin <= k <= kmax but is unknown, 2 kmax
log(n - kmin) measurements is sufficient. Furthermore, this number of
measurements is also sufficient for detection of the sparsity pattern (support)
of the vector with measurement errors provided the signal-to-noise ratio (SNR)
scales to infinity. The scaling 2 k log(n - k) exactly matches the number of
measurements required by the more complex lasso method for signal recovery with
a similar SNR scaling.Comment: 11 pages, 2 figure
Vector Approximate Message Passing for the Generalized Linear Model
The generalized linear model (GLM), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
output , arises in a range of applications such
as robust regression, binary classification, quantized compressed sensing,
phase retrieval, photon-limited imaging, and inference from neural spike
trains. When is large and i.i.d. Gaussian, the generalized
approximate message passing (GAMP) algorithm is an efficient means of MAP or
marginal inference, and its performance can be rigorously characterized by a
scalar state evolution. For general , though, GAMP can
misbehave. Damping and sequential-updating help to robustify GAMP, but their
effects are limited. Recently, a "vector AMP" (VAMP) algorithm was proposed for
additive white Gaussian noise channels. VAMP extends AMP's guarantees from
i.i.d. Gaussian to the larger class of rotationally invariant
. In this paper, we show how VAMP can be extended to the GLM.
Numerical experiments show that the proposed GLM-VAMP is much more robust to
ill-conditioning in than damped GAMP
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
The replica method is a non-rigorous but well-known technique from
statistical physics used in the asymptotic analysis of large, random, nonlinear
problems. This paper applies the replica method, under the assumption of
replica symmetry, to study estimators that are maximum a posteriori (MAP) under
a postulated prior distribution. It is shown that with random linear
measurements and Gaussian noise, the replica-symmetric prediction of the
asymptotic behavior of the postulated MAP estimate of an n-dimensional vector
"decouples" as n scalar postulated MAP estimators. The result is based on
applying a hardening argument to the replica analysis of postulated posterior
mean estimators of Tanaka and of Guo and Verdu.
The replica-symmetric postulated MAP analysis can be readily applied to many
estimators used in compressed sensing, including basis pursuit, lasso, linear
estimation with thresholding, and zero norm-regularized estimation. In the case
of lasso estimation the scalar estimator reduces to a soft-thresholding
operator, and for zero norm-regularized estimation it reduces to a
hard-threshold. Among other benefits, the replica method provides a
computationally-tractable method for precisely predicting various performance
metrics including mean-squared error and sparsity pattern recovery probability.Comment: 22 pages; added details on the replica symmetry assumptio
Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization
Generalized Linear Models (GLMs), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
arise in a range of applications in nonlinear
filtering and regression. Approximate Message Passing (AMP) methods, based on
loopy belief propagation, are a promising class of approaches for approximate
inference in these models. AMP methods are computationally simple, general, and
admit precise analyses with testable conditions for optimality for large i.i.d.
transforms . However, the algorithms can easily diverge for general
. This paper presents a convergent approach to the generalized AMP
(GAMP) algorithm based on direct minimization of a large-system limit
approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a
double-loop procedure, where the outer loop successively linearizes the LSL-BFE
and the inner loop minimizes the linearized LSL-BFE using the Alternating
Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP,
is similar in structure to the original GAMP method, but with an additional
least-squares minimization. It is shown that for strictly convex, smooth
penalties, ADMM-GAMP is guaranteed to converge to a local minima of the
LSL-BFE, thus providing a convergent alternative to GAMP that is stable under
arbitrary transforms. Simulations are also presented that demonstrate the
robustness of the method for non-convex penalties as well
- …